Optimized control system for aggregation of multiple broadband connections over radio interfaces

ABSTRACT

In accordance with embodiments disclosed herein, there are provided apparatus, systems and methods for implementing and using optimized control systems for aggregation of multiple broadband connections over radio interfaces. For example, such a system may include: a processor and a memory to perform instructions embodied by the system; a plurality of antennas; a traffic coordinator to interface to two or more wireless communications nodes together, through the system, in which each of the wireless communications nodes have access to a wide Area Network (WAN) backhaul connection independent of the system; a first wireless communications interface to a first wireless communication node established via a first of the plurality of antennas, the first wireless communications node having access to a first WAN backhaul connection; a second wireless communications interface to a second wireless communications node established via a second of the plurality of antennas, the second wireless communications node having access to a second WAN backhaul connection distinct from the first WAN backhaul connection; and a control module to receive information on traffic flows through the system and a radio environment within which the system operates, in which the control module to: issue commands to control the formation and continuation of connections of the first and second wireless communications interfaces to WAN connections and WAN backhaul connections, and to further provide scheduling and routing instructions for the WAN connections and WAN backhaul connections. Other related embodiments are disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of previously filed and copendingpatent application Ser. No. 14/431,774, entitled “OPTIMIZED CONTROLSYSTEM FOR AGGREGATION OF MULTIPLE BROADBAND CONNECTIONS OVER RADIOINTERFACES,” naming as inventors Kenneth J. Kerpez and Mung Chiang, andfiled 2015 Mar. 27, which is a U.S. National Phase Application Under 35U.S.C. § 371 of International Patent Application No. PCT/US2012/058157,entitled “OPTIMIZED CONTROL SYSTEM FOR AGGREGATION OF MULTIPLE BROADBANDCONNECTIONS OVER RADIO INTERFACES,” naming as inventors Kenneth J.Kerpez and Mung Chiang, and filed 2012 Sep. 29, which applications arehereby incorporated herein by reference in their entireties.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

TECHNICAL FIELD

The subject matter described herein relates generally to the field ofcomputing, and more particularly, to apparatus, systems and methods forimplementing and using optimized control systems for aggregation ofmultiple broadband connections over radio interfaces.

BACKGROUND

The subject matter discussed in the background section should not beassumed to be prior art merely as a result of its mention in thebackground section. Similarly, a problem mentioned in the backgroundsection or associated with the subject matter of the background sectionshould not be assumed to have been previously recognized in the priorart. The subject matter in the background section merely representsdifferent approaches, which in and of themselves may also correspond toembodiments of the claimed subject matter.

In computer networking, a wireless access point (WAP) is a device thatallows wireless devices to connect to a wired network using Wi-Fi,Bluetooth or other related standards. The wireless access point usuallyconnects to a router or operates as a router itself.

Wireless access points are commonplace, however, conventional offeringsof such wireless access points fail to operate in the most efficientmanner possible, and may be improved upon in a multitude of ways.

The present state of the art may therefore benefit from apparatuses,systems and methods for implementing and using optimized control systemsfor aggregation of multiple broadband connections over radio interfacesas described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example, and not by way oflimitation, and will be more fully understood with reference to thefollowing detailed description when considered in connection with thefigures in which:

FIG. 1A illustrates an exemplary architecture in which embodiments mayoperate;

FIG. 2A shows a diagrammatic representation of a system in whichembodiments may operate, be installed, integrated, or configured;

FIG. 2B shows an alternative diagrammatic representation of a system inwhich embodiments may operate, be installed, integrated, or configured;

FIG. 2C shows an alternative diagrammatic representation of a system inwhich embodiments may operate, be installed, integrated, or configured;

FIG. 2D shows an alternative diagrammatic representation of a system inwhich embodiments may operate, be installed, integrated, or configured;

FIG. 3A is a flow diagram illustrating a method for implementing andusing optimized control systems for aggregation of multiple broadbandconnections over radio interfaces in accordance with describedembodiments;

FIG. 3B shows an alternative diagrammatic representation of a BACKcontrol plane in accordance with which embodiments may operate;

FIG. 3C shows an alternative diagrammatic representation of wirelesscommunications interfaces in accordance with which embodiments mayoperate; and

FIG. 4 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system, in accordance with one embodiment.

DETAILED DESCRIPTION

Described herein are apparatus, systems and methods for implementing andusing optimized control systems for aggregation of multiple broadbandconnections over radio interfaces.

In accordance with one embodiment, an exemplary system may include: aprocessor and a memory to perform instructions embodied by the system; aplurality of antennas; a traffic coordinator to interface to two or morewireless communications nodes together, through the system, in whicheach of the wireless communications nodes have access to a wide AreaNetwork (WAN) backhaul connection independent of the system; a firstwireless communications interface to a first wireless communication nodeestablished via a first of the plurality of antennas, the first wirelesscommunications node having access to a first WAN backhaul connection; asecond wireless communications interface to a second wirelesscommunications node established via a second of the plurality ofantennas, the second wireless communications node having access to asecond WAN backhaul connection distinct from the first WAN backhaulconnection; and a control module to receive information on traffic flowsthrough the system and a radio environment within which the systemoperates, in which the control module to: issue commands to control theformation and continuation of connections of the first and secondwireless communications interfaces to WAN connections and WAN backhaulconnections, and to further provide scheduling and routing instructionsfor the WAN connections and WAN backhaul connections.

In the following description, numerous specific details are set forthsuch as examples of specific systems, languages, components, etc., inorder to provide a thorough understanding of the various embodiments. Itwill be apparent, however, to one skilled in the art that these specificdetails need not be employed to practice the disclosed embodiments. Inother instances, well known materials or methods have not been describedin detail in order to avoid unnecessarily obscuring the disclosedembodiments.

In addition to various hardware components depicted in the figures anddescribed herein, embodiments further include various operations whichare described below. The operations described in accordance with suchembodiments may be performed by hardware components or may be embodiedin machine-executable instructions, which may be used to cause ageneral-purpose or special-purpose processor programmed with theinstructions to perform the operations. Alternatively, the operationsmay be performed by a combination of hardware and software, includingsoftware instructions that perform the operations described herein viamemory and one or more processors of a computing platform.

Embodiments also relate to a system or apparatus for performing theoperations herein. The disclosed system or apparatus may be speciallyconstructed for the required purposes, or it may comprise a generalpurpose computer selectively activated or reconfigured by a computerprogram stored in the computer. Such a computer program may be stored ina non-transitory computer readable storage medium, such as, but notlimited to, any type of disk including floppy disks, optical disks,flash, NAND, solid state drives (SSDs), CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring non-transitory electronic instructions, each coupled to acomputer system bus. In one embodiment, a non-transitory computerreadable storage medium having instructions stored thereon, causes oneor more processors within an apparatus to perform the methods andoperations which are described herein. In another embodiment, theinstructions to perform such methods and operations are stored upon anon-transitory computer readable medium for later execution.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus nor are embodimentsdescribed with reference to any particular programming language. It willbe appreciated that a variety of programming languages may be used toimplement the teachings of the embodiments as described herein.

FIG. 1 illustrates an exemplary architecture 100 in which embodimentsmay operate. Asymmetric Digital Subscriber Line (ADSL) systems (one formof Digital Subscriber Line (DSL) systems), which may or may not includesplitters, operate in compliance with the various applicable standardssuch as ADSL1 (G.992.1), ADSL-Lite (G.992.2), ADSL2 (G.992.3),ADSL2-Lite G.992.4, ADSL2+(G.992.5) and the G.993.x emergingVery-high-speed Digital Subscriber Line or Very-high-bitrate DigitalSubscriber Line (VDSL) standards, as well as the G.991.1 and G.991.2Single-Pair High-speed Digital Subscriber Line (SHDSL) standards, allwith and without bonding, and/or the G.997.1 standard (also known asG.ploam).

In accordance with embodiments described herein, end-user consumers,including residential consumers and business consumers, may connect tothe Internet by way of a Wide Area Network (WAN) backhaul connection toa Service Provider (SP), such as an Internet Service Provider (ISP), orto a Service Provider that provides one or more of data connectivity,voice connectivity, video connectivity, and mobile device connectivityto a plurality of subscribers. Such Service Providers may include aDigital Subscriber Line (DSL) internet service provider which providesits subscribing end-users with Internet bandwidth at least partiallyover copper twisted pair telephone lines, such as that conventionallyutilized to carry analog telephone service (e.g., Plain Old TelephoneService (POTS); a coaxial cable internet service provider which providesend-users with Internet bandwidth at least partially over coaxial cable,such as that conventionally utilized to carry “cable” televisionsignals; or a fiber optics internet service provider which providesend-users with Internet bandwidth at over fiber optic cable thatterminates at a customer's premises. Other variants exist as well, suchas ISPs which provide Internet bandwidth as an analog signal over ananalog telephone based connection, ISPs that provide Internet bandwidthover a one-way or two-way satellite connection, and ISPs that provideInternet bandwidth at least partially over power lines, such as powerlines conventionally utilized to transmit utility power (e.g.,electricity) to an end-user's premises, or ISPs that provide Internetbandwidth at least partially over wireless channels, such as wireless(e.g., WiFi) connectivity at hotspots, or mobile data connectivity viatechnologies and standards such as WiMax, 3G/4G, LTE, etc.

In performing the disclosed functions, systems may utilize a variety ofoperational data (which includes performance data) that is available atan Access Node (AN).

In FIG. 1, user's terminal equipment 102 (e.g., a Customer PremisesEquipment (CPE) device or a remote terminal device, network node, LANdevice, etc.) is coupled to a home network 104, which in turn is coupledto a Network Termination (NT) Unit 108. DSL Transceiver Units (TU) arefurther depicted (e.g., a device that provides modulation on a DSL loopor line). In one embodiment, NT unit 108 includes a TU-R (TU Remote),122 (for example, a transceiver defined by one of the ADSL or VDSLstandards) or any other suitable network termination modem, transceiveror other communication unit. NT unit 108 also includes a ManagementEntity (ME) 124. Management Entity 124 can be any suitable hardwaredevice, such as a microprocessor, microcontroller, or circuit statemachine in firmware or hardware, capable of performing as required byany applicable standards and/or other criteria. Management Entity 124collects and stores, among other things, operational data in itsManagement Information Base (MIB), which is a database of informationmaintained by each ME capable of being accessed via network managementprotocols such as Simple Network Management Protocol (SNMP), anadministration protocol used to gather information from a network deviceto provide to an administrator console/program or via TransactionLanguage 1 (TL1) commands, TL1 being a long-established command languageused to program responses and commands between telecommunication networkelements.

Each TU-R 122 in a system may be coupled with a TU-C (TU Central) in aCentral Office (CO) or other central location. TU-C 142 is located at anAccess Node (AN) 114 in Central Office 146. A Management Entity 144likewise maintains an MIB of operational data pertaining to TU-C 142.The Access Node 114 may be coupled to a broadband network 106 or othernetwork, as will be appreciated by those skilled in the art. TU-R 122and TU-C 142 are coupled together by a loop 112, which in the case ofADSL may be a twisted pair line, such as a telephone line, which maycarry other communication services besides DSL based communications.

Several of the interfaces shown in FIG. 1 are used for determining andcollecting operational data. The Q interface 126 provides the interfacebetween the Network Management System (NMS) 116 of the operator and ME144 in Access Node 114. Parameters specified in the G.997.1 standardapply at the Q interface 126. The near-end parameters supported inManagement Entity 144 may be derived from TU-C 142, while far-endparameters from TU-R 122 may be derived by either of two interfaces overthe UA interface. Indicator bits and EOC messages may be sent usingembedded channel 132 and provided at the Physical Medium Dependent (PMD)layer, and may be used to generate the required TU-R 122 parameters inME 144. Alternately, the Operations, Administration and Maintenance(OAM) channel and a suitable protocol may be used to retrieve theparameters from TU-R 122 when requested by Management Entity 144.Similarly, the far-end parameters from TU-C 142 may be derived by eitherof two interfaces over the U-interface. Indicator bits and EOC messageprovided at the PMD layer may be used to generate the required TU-C 142parameters in Management Entity 124 of NT unit 108. Alternately, the OAMchannel and a suitable protocol may be used to retrieve the parametersfrom TU-C 142 when requested by Management Entity 124.

At the U interface (also referred to as loop 112), there are twomanagement interfaces, one at TU-C 142 (the U-C interface 157) and oneat TU-R 122 (the U-R interface 158). Interface 157 provides TU-C 142near-end parameters for TU-R 122 to retrieve over the U interface/loop112. Similarly, U-R interface 158 provides TU-R near-end parameters forTU-C 142 to retrieve over the U interface/loop 112. The parameters thatapply may be dependent upon the transceiver standard being used (forexample, G.992.1 or G.992.2). The G.997.1 standard specifies an optionalOperation, Administration, and Maintenance (OAM) communication channelacross the U interface. If this channel is implemented, TU-C and TU-Rpairs may use it for transporting physical layer OAM messages. Thus, theTU transceivers 122 and 142 of such a system share various operationaldata maintained in their respective MIBs.

Depicted within FIG. 1 is apparatus 170 operating at various optionallocations in accordance with several alternative embodiments. Forexample, in accordance with one embodiment, apparatus 170 is locatedwithin home network 104, such as within a LAN. In one embodimentapparatus 170 operates as a DSL modem, such as a Customer Premises (CPE)modem. In another embodiment, apparatus 170 operates as a controllercard or as a chipset within a user's terminal equipment 102 (e.g., aCustomer Premises Equipment (CPE) device or a remote terminal device,network node, LAN device, etc.) coupled to the home network 104 asdepicted. In another embodiment, apparatus 170 operates as a separateand physically distinct stand alone unit which is connected between theuser's terminal equipment 102 and a DSL line or loop. In one embodiment,apparatus 170 operates within an Access Point (AP), within a WirelessAccess Point (WAP), or within a router (e.g., a WiFi router or otherwireless technology router). In one embodiment, apparatus 170 embodies aBroadband AP Control Keeper or “BACK” as is described herein.

As used herein, the terms “user,” “subscriber,” and/or “customer” referto a person, business and/or organization to which communicationservices and/or equipment are and/or may potentially be provided by anyof a variety of service provider(s). Further, the term “customerpremises” refers to the location to which communication services arebeing provided by a service provider. For example, Public SwitchedTelephone Network (PSTN) used to provide DSL services to customerpremises are located at, near and/or are associated with the networktermination (NT) side of the telephone lines. Example customer premisesinclude a residence or an office building.

As used herein, the term “service provider” refers to any of a varietyof entities that provide, sell, provision, troubleshoot and/or maintaincommunication services and/or communication equipment. Example serviceproviders include a telephone operating company, a cable operatingcompany, a wireless operating company, an internet service provider, orany service that may independently or in conjunction with a broadbandcommunications service provider offer services that diagnose or improvebroadband communications services (DSL, DSL services, cable, etc.).

Additionally, as used herein, the term “DSL” refers to any of a varietyand/or variant of DSL technology such as, for example, Asymmetric DSL(ADSL), High-speed DSL (HDSL), Symmetric DSL (SDSL), and/or Veryhigh-speed/Very high-bit-rate DSL (VDSL). Such DSL technologies arecommonly implemented in accordance with an applicable standard such as,for example, the International Telecommunications Union (I.T.U.)standard G.992.1 (a.k.a. G.dmt) for ADSL modems, the I.T.U. standardG.992.3 (a.k.a. G.dmt.bis, or G.ads12) for ADSL2 modems, I.T.U. standardG.992.5 (a.k.a. G.ads12plus) for ADSL2+ modems, I.T.U. standard G.993.1(a.k.a. G.vds1) for VDSL modems, I.T.U. standard G.993.2 for VDSL2modems, I.T.U. standard G.994.1 (G.hs) for modems implementinghandshake, and/or the I.T.U. G.997.1 (a.k.a. G.ploam) standard formanagement of DSL modems.

References to connecting a DSL modem and/or a DSL communication serviceto a customer are made with respect to exemplary Digital Subscriber Line(DSL) equipment, DSL services, DSL systems and/or the use of ordinarytwisted-pair copper telephone lines for distribution of DSL services andit shall be understood that the disclosed methods and apparatus tocharacterize and/or test a transmission medium for communication systemsdisclosed herein may be applied to many other types and/or variety ofcommunication equipment, services, technologies and/or systems. Forexample, other types of systems include wireless distribution systems,wired or cable distribution systems, coaxial cable distribution systems,Ultra High Frequency (UHF)/Very High Frequency (VHF) radio frequencysystems, satellite or other extra-terrestrial systems, cellulardistribution systems, broadband power-line systems and/or fiber opticnetworks. Additionally, combinations of these devices, systems and/ornetworks may also be used. For example, a combination of twisted-pairand coaxial cable interfaced via a balun connector, or any otherphysical-channel-continuing combination such as an analog fiber tocopper connection with linear optical-to-electrical connection at anOptical Network Unit (ONU) may be used.

The phrases “coupled to,” “coupled with,” connected to,” “connectedwith” and the like are used herein to describe a connection between twoelements and/or components and are intended to mean coupled/connectedeither directly together, or indirectly, for example via one or moreintervening elements or via a wired/wireless connection. References to a“communication system” are intended, where applicable, to includereference to any other type of data transmission system.

FIG. 2 shows a diagrammatic representation of a system 200 in whichembodiments may operate, be installed, integrated, or configured,including various components of such a system 200 interconnected via abus 215 communication means.

According to one embodiment, such a system 200 includes a processor 290and a memory 295 to perform instructions embodied by the system 200. Insuch an embodiment, the system 200 further includes a plurality ofantennas 211A and 211B and a traffic coordinator 220 to interface to twoor more wireless communications nodes 299A and 299B together, throughthe system 200, in which each of the wireless communications nodes haveaccess to a wide Area Network (WAN) backhaul connection 298A and 298Bindependent of the system 200. For example, the wireless communicationnodes 299A and 299B are depicted as being indirectly interfaced as notedby element 297, or stated differently, they are interfaced together notby communicating directly with each other, but rather, by communicatingthrough an intermediary, depicted here as system 200. In thisembodiment, each of the depicted wireless communication nodes 299A and299B has access to a WAN backhaul as depicted by elements 298A and 298B.Notably, the WAN backhaul 298A and 298B connections are accessible tothe respective wireless communication nodes 299A and 299B without havingto rely upon the system 200, and thus, the WAN backhaul 298A and 298Bare said to be independent of the system 200.

In such an embodiment, the system 200 further includes: a first wirelesscommunications interface 212A to a first wireless communication node299A established via a first of the plurality of antennas 211A, thefirst wireless communications node having access to a first WAN backhaulconnection 298A and a second wireless communications interface 212B to asecond wireless communications node 299B established via a second of theplurality of antennas 211B, the second wireless communications nodehaving access to a second WAN backhaul connection 298B distinct from thefirst WAN backhaul connection 298A.

According to such an embodiment, the system 200 further includes acontrol module 260 to receive information 222 on traffic flows 221through the system 200 and a radio environment 250 within which thesystem 200 operates.

According to such an embodiment, the control module 260 issues commands223 to control the formation and continuation of connections (e.g., thewireless communication interfaces 212A and 212B) of the first and secondwireless communications interfaces to the WAN connections and WANbackhaul connections (e.g., 298A and 298B), and the control module 260further provides scheduling and routing instructions 224 for the WANconnections and WAN backhaul connections (e.g., 298A and 298B).

According to one embodiment, the system 200 embodies a “Broadband Accesspoint Control Keeper system,” a “B.A.C.K. System,” a “BACK system,” or a“BACK device.” According to one embodiment, the apparatus or BACK devicedepicted at element 170 of FIG. 1 is embodied within such a system 200.

According to one embodiment, the control module 260 is embodied withinsuch a BACK system, in which the BACK system controls settings at thefirst wireless communications node 299A, controls settings at the secondwireless communications node 299B or controls settings at both the firstand second wireless communications nodes 299A and 299B, in which thesettings are selected from the following: radio link connection settingsaffecting the respective first or second first wireless communicationsinterface 212A or 212B; channel assignments affecting the respectivefirst or second first wireless communications interface 212A or 212B;broadband connection settings affecting the respective first or secondWAN backhaul connection 298A or 298B; connection assignments amongnetwork stations (STAs), network Access Points (APs), and broadbandbackhaul connections at the STAs and/or APs through which access to therespective first or second WAN backhaul connection 298A or 298B isprovided; Internet Protocol (IP) address assignments for the flow ofdata packets 221; IP address assignments for a first and a secondsub-set of the flow of data packets 221; Quality of Service (QoS)classifications for the flow of data packets 221; QoS classificationsfor the respective first and second sub-sets of flows; QoS throttlingparameters for the flow of data packets, the respective first and secondsub-sets of flows 221, or both; routing of the respective first andsecond sub-sets of flows 221 according to available WAN backhaulconnections 298A and 298B and timeslots on the available WAN backhaulconnections 298A and 298B; load balancing parameters affecting the flowof data packets 221, the respective first and second sub-sets of flows221, or both; and fairness criteria for all traffic processed by thefirst wireless communication node 299A, the second wirelesscommunication node 299B or both the first and second wirelesscommunication nodes 299A and 299B.

According to one embodiment, the first wireless communications node(299A) is embodied within a network router, in which the network routerestablishes connectivity to the first WAN backhaul connection 298A, andfurther in which the system 200 establishes access to the first WANbackhaul connection 298A through the first wireless communicationsinterface 212A to the network router.

According to one embodiment, the first wireless communications node(299A) is embodied within a modem directly interfaced to the first WANbackhaul connection 298A, in which the system 200 establishes access tothe first WAN backhaul connection 298A through the modem.

According to one embodiment, a flow of data packets through the system200 is managed by the traffic coordinator 220 of the system 200 suchthat a first sub-set of the flow (e.g., some but not all of 221) isrouted through the first WAN backhaul connection 298A and a secondsub-set of the flow is routed through the second WAN backhaul connection298B.

According to another embodiment, the flow of data packets 221 throughthe system 200 managed by the traffic coordinator 220 constitutes thetraffic coordinator 220 managing the flow of data packets 221 byapportioning time-slots of the respective first or second WAN backhaulconnection 298A-B to carry the respective first or second sub-set of theflow 221.

According to another embodiment, each respective first or second sub-setof the flow of data packets 221 is allocated by the traffic coordinator220 of the system 200 to be serviced by one of the first or second WANbackhaul connections 298A-B on the basis of: traffic associated with anapplication; traffic associated with an interface; traffic associatedwith a service designation; and traffic associated with a Quality ofService (QoS) tag.

According to another embodiment, the first and second wirelesscommunications interfaces 212A-B with the system 200 arefrequency-multiplexed, each of the first and second wirelesscommunications interfaces 212A-B being associated with separatefrequency bands managed by the system 200. For example, the separatefrequency bands may be dictated by the traffic coordinator 220 of thesystem 200. In such an embodiment, the system 200 further provides anaggregated WAN backhaul connection via the first and second wirelesscommunications interfaces 211A-B to the respective first and second WANbackhaul connections 298A-B using the frequency bands as managed by thesystem 200. Unlike time division, frequency channels may overlapsomewhat, at least in the roll-off.

According to another embodiment, the first and second wirelesscommunications interfaces 212A-B with the system 202 aretime-multiplexed, each of the first and second wireless communicationsinterfaces 212A-B being associated with non-overlapping time-slotsmanaged by the system. According to such an embodiment, the system 200further provides an aggregated WAN backhaul connection through the firstand second wireless communications interfaces 212A-B to the respectivefirst and second WAN backhaul connections 298A-B using thenon-overlapping time slots as managed by the system 200.

According to one embodiment, such time-slots are strictlynon-overlapping with one another, distinguished from thefrequency-multiplexed having frequency channels that may overlap.According to one embodiment, the non-overlapping time-slots are furthercharacterized insomuch that each has at least some guard-time betweenthem.

According to one embodiment, the flow of packets 221 is managed byallocating the first sub-set of the flow 221 to time-slots carried bythe first WAN backhaul connection 298A and further by allocating thesecond sub-set of the flow 221 to time-slots carried by the second WANbackhaul connection 298B.

FIG. 2B shows an alternative diagrammatic representation of a system 201in which embodiments may operate, be installed, integrated, orconfigured.

According to one embodiment, the first wireless communications node(e.g., 299A at FIG. 2A) is embodied within a wireless Access Point(wireless AP) 293A, in which the wireless AP 293A establishes a LocalArea Network (LAN) 285A for one or more nodes 292A, 292B, 292Ccommunicatively interfaced thereto; and further in which the system 201communicates with and controls a node 292A within the LAN 285A. In suchan embodiment, the system 201 establishes access to the first WANbackhaul connection 298A through its communication and control with thenode 292A within the LAN 285A.

FIG. 2C shows an alternative diagrammatic representation of a system 202in which embodiments may operate, be installed, integrated, orconfigured.

According to another embodiment, the second wireless communications node(e.g., 299B at FIG. 2A) is embodied within a second wireless AP 293B, inwhich the second wireless AP 293B establishes a second LAN 285B,distinct from the first LAN 285A, for one or more nodes 292D, 292E, and292F, communicatively interfaced thereto; and further in which thesystem 202 communicates with and controls a node 292D within the secondLAN 285B while simultaneously communicating with and controlling thenode 292A within the first LAN 285A. In such an embodiment, the system202 establishes access to the second WAN backhaul connection 298Bthrough its participation as a node (one of 292D-F) within the secondLAN 285B.

According to one embodiment, the first wireless communications node(e.g., 299A of FIG. 2A or 293A of FIG. 2C) is embodied within a wirelessstation operating as a peer node within a Local Area Network (LAN) 285A,in which the peer node has access to the first WAN backhaul connection298A via the LAN 285A, and further in which the first wirelesscommunications interface 121A is a peer-to-peer connection with the peernode. In such an embodiment, the system 202 establishes access to thefirst WAN backhaul connection 298A through the peer-to-peer connectionwith the peer node (e.g., wireless access point 293A operating as a nodewithin LAN 285A).

According to one embodiment, functionality of the control module 260 forthe system 200 is distributed across one or more physical devicesselected from the list including: a remote server; the first wirelesscommunications device (e.g., 299A of FIG. 2A or 293A of FIG. 2C); thesecond wireless communications device (e.g., elements 299B or 293B); thefirst wireless communications node 292A; the second wirelesscommunications node 292B; a router; a switch; and a broadbandaggregation device.

According to one embodiment, each of the first wireless communicationsnode 292A and the second wireless communications node 292B are selectedfrom the group of devices including: a third generation (3G) compatibledevice; a fourth generation (4G) compatible device; a Long TermEvolution (LTE) compatible device; an access point; a modem; a router; agateway; a Digital Subscriber Line (DSL) Customer Premises Equipment(CPE) modem; an in-home power line device; a Home Phoneline NetworkAlliance (HPNA) based device; an in-home coax distribution device; aG.hn compatible device; an in-home metering communication device; anin-home appliance communicatively interfaced with the LAN; a wirelessfemtocell base station; a wireless compatible base station; a wirelessmobile device repeater; a wireless mobile device base station; anEthernet gateway; a computing device connected to the LAN; a HomePlugdevice; an IEEE P1901 standards compatible access Broadband over PowerLine (BPL) device; an Ethernet connected computer peripheral device; anEthernet connected router; an Ethernet connected wireless bridge; anEthernet connected network bridge; and an Ethernet connected networkswitch.

FIG. 2D shows an alternative diagrammatic representation of a system 203in which embodiments may operate, be installed, integrated, orconfigured.

According to one embodiment, such a system 203 further includes a thirdwireless communications interface 212C to a third wirelesscommunications node 292G, in which the third wireless communicationsnode 292G has access to a third WAN backhaul connection 298C distinctfrom the first and the second WAN backhaul connections 298A-B.

According to one embodiment, the system 203 further includes a backhaulassessment module 265. In one embodiment, the backhaul assessment module265 is operable to perform the following operations: (a) measureperformance of connectivity through the first, second, and thirdwireless communications interfaces (212A, 212B, and 212C) to therespective first, second, and third WAN backhaul connections (298A,298B, and 298C), and further operable to (b) select two or more of theavailable WAN backhaul connections (212A, 212B, and 212C) to service theflow of data packets 221.

According to another embodiment, the backhaul assessment module 265 isoperable to: (a) measure performance of connectivity through allavailable wireless communications interfaces 212A-C, and (b) furtheroperable to select two or more of the available wireless communicationsinterfaces 212A-C to service the flow of data packets 221 on the basisof: a WAN backhaul connection type preference associated with theassessed wireless communications interfaces (e.g., certain connectiontypes may be specified as preferable over others, such as WiFipreferable over LTE, 3G, 4G, etc. regardless of speed, congestion,etc.); further operable to select two or more of the available wirelesscommunications interfaces 212A-C on the basis of a number of node hopsbetween the system and the servicing WAN backhaul connection (e.g.,indirect connections may be less preferred, etc.); further operable toselect two or more of the available wireless communications interfaces212A-C on the basis of an assessed signal strength of the assessedwireless communications interfaces 212A-C; further operable to select onthe basis of assessed traffic congestion at the assessed wirelesscommunications interfaces 212A-C, at the corresponding WAN backhaulinterface 298A-C, or both; and further operable to select on the basisof assessed available capacity at the assessed wireless communicationsinterface 212A-C, at the corresponding WAN backhaul interface 298A-C, orboth.

According to another embodiment, the system 203 includes means tocommunicate with and control the WAN backhaul (any of 298A-C) from thesystem 203. For instance, a DSM system, DSL management system,management device, etc., may be utilized in conjunction with thewireless control system so as to control and manipulate the WAN backhaulconnection in the same manner that the wireless or WiFi connections arecontrolled and manipulated, thus providing even further overall signaland connectivity enhancements.

FIG. 3A is a flow diagram illustrating a method 300 for implementing andusing optimized control systems for aggregation of multiple broadbandconnections over radio interfaces in accordance with describedembodiments. Method 300 may be performed by processing logic that mayinclude hardware (e.g., circuitry, dedicated logic, programmable logic,microcode, etc.), software (e.g., instructions run on a processingdevice to perform various operations such as interfacing, managing,receiving, controlling, analyzing, collecting, generating, monitoring,diagnosing, or some combination thereof). In one embodiment, method 300is performed or coordinated via an apparatus such as that depicted atelement 170 of FIG. 1 or the system 200 at FIG. 2A (e.g., a B.A.C.K.system) and described throughout. Some of the blocks and/or operationslisted below are optional in accordance with certain embodiments. Thenumbering of the blocks presented is for the sake of clarity and is notintended to prescribe an order of operations in which the various blocksmust occur.

Method 300 begins with processing logic for establishing a firstwireless communications interface to a first wireless communicationsnode, in which the first wireless communications node has access to afirst Wide Area Network (WAN) backhaul connection (block 305).

At block 310, processing logic establishes a second wirelesscommunications interface to a second wireless communications node, inwhich the second wireless communications node has access to a second WANbackhaul connection distinct from the first WAN backhaul connection.

At block 315, processing logic manages a flow of data packets such thata first sub-set of the flow is transmitted through the first WANbackhaul connection and a second sub-set of the flow is transmittedthrough the second WAN backhaul connection.

At block 320, processing logic collects and analyzes information ontraffic and a radio environment from a plurality of network elements ormanagement systems. For example, the network elements or managementsystems may be any of the nodes, wireless communication nodes,peer-nodes, routers, etc., as described above.

At block 325, processing logic controls the establishment andcontinuation of connections of the first and second wirelesscommunications interfaces to WAN connections and WAN backhaulconnections based on the collected information and analyses.

According to another embodiment of the preceding method 300, commandsare issued to provide scheduling and routing instructions for the WANconnections and the WAN backhaul connections.

In yet another embodiment of the method, there are further operationsincluding: optimizing the flow of data packets through the first andsecond respective WAN backhaul connections according to a schedulingalgorithm, a load balancing algorithm, or both.

For instance, multiple nodes or stations (STAs) may each be providedwith a utility function chosen either by a WiFi provider or by aconsumer. Multiple Access Points (APs) further may each be provided witha backhaul capacity that may vary by backhaul provider.

According to one embodiment, an algorithm will vary the fraction of timeSTA k spends connecting to AP i over a short timescale. This is thescheduling decision. In one embodiment, the fractions must sum up acrossall APs to be less than 1 for each STA. There is a link capacity fromeach STA to each AP. These link capacities must be collectivelyfeasible. The throughput from each STA to an AP is the product of ascheduling decision and the link capacity.

According to one embodiment, such a BACK system maximizes the sum ofutility functions, one per STA as a function of the total throughput(e.g., the sum of per-AP link throughput for that STA). Thisoptimization may be carried out under several constraints. In additionto the scheduling constraint and the link capacity constraint above,another constraint may be that the sum of the throughputs across allSTAs connected to each AP cannot be bigger than the backhaul capacity ofthat AP.

Fairness can be controlled through the choice of utility functions forthe STAs. For example, proportional fairness can be obtained by usinglogarithmic utility functions. Furthermore, weights can be installed infront of each utility function. These weights can be derived from eitherbilling differences (e.g., some users pay more for greater weightedpreference), or from multiple QoS classes, including in-building users,users passing by, users of various degrees of quality assurance, and soforth. These weights can also reflect the number of parallel TCPsessions for a given application flow.

Another issue considered by aspects of fairness control is therelationship between the rate that a STA would receive using a single APand the rate it receives when using multiple APs. The ratio betweenthese two rates may be controlled by a BACK system as described.

This problem can be solved in short timescales approximately (pertimeslot), or over longer timescale for a target equilibrium. It can besolved in various ways, but use of extended information and control of aprovider-supported control plane can dramatically enhance the efficiencyof solving this problem. Furthermore, if the backhaul provider(s)participate in the optimization, then {B_i} becomes variables too.

Such a BACK system may directly relay these optimized vectors to eachSTA and AP, then the equipment assigns each transmission a path andtimeslot such that the total traffic matches the optimal schedules andthroughputs as close as possible.

Alternatively, the BACK system may indirectly assign link parameters andcapacity to approximate the optimal solution. Differentsource/destination addresses, or different flows (i.e., video streams)may be assigned over different paths. Or a flow may be broken intomultiple fragments, with a tracker file created by the BACK whichdetermines the paths and slots that each data fragment is sent over.

In the special case of each STA being scheduled only to one AP at atime, solving this problem amounts to switching APs. In addition tosolving the above problem formulation, we can further impose twoadditional elements to the solution method: Randomization andHysteresis.

With randomization, each STA decides to switch from one AP to anotherwith a certain probability so that the chance of simultaneous switchingis smaller.

With hysteresis, switching from a STA scheduled to AP1 to the STAscheduled to AP2 means that the chance of switching back to AP1 within afew timeslots is lower, so as to avoid noise-induced flip-flop or athrashing condition among APs.

According to another embodiment, the method further includes operationsfor issuing optimization instructions to the first wirelesscommunications node or the second wireless communications node, or both,to implement configuration parameters in fulfillment of a determinedscheduling and load balancing strategy.

According to another embodiment, the optimization instructions arebased, at least in part, on one or more performance metrics retrievedfrom the first wireless communications node or the second wirelesscommunications node, or both.

According to another embodiment, issuing the optimization instructionsincludes iteratively issuing the optimization instructions to improvemeasured performance, in which each of a plurality of iterationsincludes at least: (a) retrieving one or more performance metrics fromthe first wireless communications node or the second wirelesscommunications node, or both; (b) evaluating the retrieved performancemetrics; (c) determining updated configuration parameters in fulfillmentof an updated scheduling and load balancing strategy; and (d) issuingupdated optimization instructions to the first wireless communicationsnode or the second wireless communications node, or both, to implementthe updated configuration parameters.

According to a related embodiment, each of the plurality of iterationsfurther includes an assessment of historic traffic data.

In one embodiment, issuing the optimization instructions includes anassessment based on one or more of: available performance tuningparameters; available historic traffic data; available historic radiolink performance data within a geographic location-aware map; availableperformance and triangulation data within the geographic location-awaremap; available selective Quality of System (QoS) parameters; availableinformation on the underlying wireless network topology; availableinformation on interference in the wireless network; bias toward one ormore reliability targets; and available rewards and incentives fordevices that participate in an aggregated WAN backhaul connection.

According to another embodiment, issuing the optimization instructionsincludes allocating increased bandwidth for the flow of data packetswithin an aggregated WAN backhaul connection based on a user allowing awireless communications interface to wireless communications node havingaccess to a WAN backhaul connection.

In accordance with one embodiment, there is a non-transitory computerreadable storage medium having instructions stored thereon that, whenexecuted by a processor of an apparatus, system, BACK device, or othercompatible embodiment of the operations described herein, theinstructions cause the apparatus to perform operations including:establishing a first wireless communications interface to a firstwireless communications node, the first wireless communications nodehaving access to a first Wide Area Network (WAN) backhaul connection;establishing a second wireless communications interface to a secondwireless communications node, the second wireless communications nodehaving access to a second WAN backhaul connection distinct from thefirst WAN backhaul connection; managing a flow of data packets such thata first sub-set of the flow is transmitted through the first WANbackhaul connection and a second sub-set of the flow is transmittedthrough the second WAN backhaul connection; collecting and analyzinginformation on traffic and a radio environment from a plurality ofnetwork elements or management systems; and controlling theestablishment and continuation of connections of the first and secondwireless communications interfaces to WAN connections and WAN backhaulconnections based on the collected information and analyses.

FIG. 3B shows an alternative diagrammatic representation of a BACKcontrol plane 399 (e.g., a system, a BACK device, or a BACK system,etc.) in accordance with which embodiments may operate. While anexemplary physical architecture is depicted, there may be many more APsand STAs than are set forth by this example.

According to one embodiment, such a BACK system or BACK control plane399 implements the methodologies set for above. For instance, such asystem controls a multitude of IEEE 802.11 devices connected overmultiple broadband backhaul connections, including Stations (STAs) suchas smart phones, tablets, laptops, desktops, game consoles, and InternetTV sets that transmit and receive in the ISM bands, and Access Points(APs) that have air-interface connections with STAs on the one side andbackhaul connection on the other to Ethernet, DSL, fiber, cable, or anyother means of connecting to the rest of the Internet. The STAs and APsmay communicate with advanced WiFi technologies, such as Super WiFi andmulti-user MIMO.

According to one embodiment, each STA can connect to multiple APs andthe associated broadband backhaul links. There are four modes ofoperation as defined by the following matrix, with acronyms indicatedbelow in Table 1 depicting the applicable connection types:

TABLE 1 Time-multiplexed Simultaneous connections connections tomultiple APs to multiple APs Direct connection from each D-TM D-S STA tomultiple APs Indirect connection, where I-TM I-S each STA connect toother STAs and then their APs

In I-TM and I-S modes, multihop radio connections among STAs arerequired, for example, via ad hoc mode in 802.11 or with dual radios.The methodologies also allow the APs to form a multihop network amongthemselves, so that bottlenecks of some backhaul capacity can be routedaround through a longer path of APs. For example, 2.4 GHz links can bebackhauled over a 40 MHz channel at 5 GHz.

In D-TM and I-TM modes, each STA at any given time only connects to asingle AP, but switches among multiple APs over time according to ascheduling vector S(t) that depends on time t, e.g., S(100)=[1 0] andS(101)=[0 1].

In D-S and I-S modes, each STA connects to multiple APs at the sametime, with traffic spread over them according to a load balancing vectorS(t) that may depend on time t, e.g., S(100)=[0.8 0.2] and S(101)=[0.50.5]. Note that in TM modes, S is a binary vector, whereas in S modes, Sis a real vector.

Which mode is in operation depends in part on the type of radio andconnection management available in a given system. Some of the describedmethods apply to all modes, while others may be targeted specificallyfor certain modes.

According to the depicted architecture 301 having the BACK control plane399 therein, there are provided four distinct broadback backhaulconnections 1-4, set forth as elements 381, 382, 383, and 384respectively. Each is connect with a corresponding access point, inwhich the broadband backhaul #1 connection 381 connects with AP1 atelement 371, the broadband backhaul #2 connection 382 connects with AP2at element 372, the broadband backhaul #3 connection 383 connects withAP3 at element 373, and in which the broadband backhaul #4 connection384 connects with AP4 at element 374. There are two stations depicted asSTA1 at element 361 and STA2 at element 362. Wireless interfaceconnections are depicted between the various access points and stations,in which AP1 371 connects with STA1 361; AP2 372 connects with both STA1361 and also AP3 373, AP3 373 being connected only with AP2 372 (andbroadband backhaul #3 at element 383); AP4 374 being connected with onlySTA2 362 (and broadband backhaul #4 at element 384); and finally STA2being connected with both STA1 361 and AP4 374.

Such Multi-AP architectures are indeed feasible. Control overhead istolerable, managing packet transition and handoff is possible,interaction with upper layer protocols such as TCP can be carried out,and security can be maintained as well, thus enabling multi-homedbroadband access. Unfortunately, no conventional system has addressedthe automated management and control functions which are necessary forhigh performance of a large scale system.

A control plane which is embodied by the Broadband AP Control Keeper(BACK) inputs measurements of the radio and backhaul environment, thecapacity of each virtual link, and the load of each STA. The BACK systemthen determines optimal control parameter settings using algorithms,thus providing optimization of the long-term architectural set-up aswell as real-time performance.

According to certain embodiments, BACK system controls include: Linksettings, such as the selection of WiFi channels used by APs. Channelselection is done to avoid interference from APs that are under thecontrol of the BACK as well as APs outside of BACK control. The goal isto use channels with the least interference, where interference isdetermined by received signal levels as well as by the traffic levels onthe channel. Channel selection is implemented by the BACK system whichassigns multiple channels and determines their traffic loads; both ofwhich affect interference.

According to certain embodiments, BACK system controls further includeconnection control. For example, each STA can connect to severalbackhaul paths directly or indirectly, using time-multiplexedconnections or simultaneous connections. Connection durations of onlytens of milliseconds are practical, and so the BACK can assignscheduling vectors, S(t), to have many short duration time-slots to manyAPs and STAs, which are chosen to avoid interference. Or staticconnections may be assigned with simple load-balancing vectors S(t), orwith simple main and back-up paths, or with S(t) only slowly varyingsuch as with time-of-day.

According to certain embodiments, BACK system controls further includecontrol of real-time traffic. Each radio connection to a backhaul linkcan be thought of as a virtual interface. Different IP addresses, flows,or even individual packets are routed over different interfaces via theoptimal traffic assignments as determined by the BACK.

Each of the control areas affects the others. Since the individualproblems do not decouple, they may be optimized collectively by the BACKsystem.

According to one embodiment, optimizing considers the following goals:(a) Multi-homed load balancing in which efficiency of the entire system,end to end, including the air interface and the broadband backhaul; (b)individual performance maximization in which efficiency of eachindividual STA and the Pareto optimal tradeoff among them; and (c)fairness of backhaul capacity allocation, of air interface capacityallocation, and of QoS for different classes of users.

Design bottlenecks of prior unsatisfactory solutions are overcome. Forinstance, there are provided: (a) incentive mechanisms, such as “tit fortat;” (b) stability of alternative path selection and reliability ofend-to-end paths; (c) minimization of message passing required amongSTAs and APs and the time to switch the wireless communication paths;(d) measurement of backhaul capacity; (e) measurement of air-interfacecapacity in a time-varying environment; and (f) measurement of radioloss to and from different STAs and APs over different locations, inwhich such measurement can leverage STA location data from GPS ortriangulation.

A transparent and optimized control plane is provided as an effectivemeans to address the above issues, through, for example, (a)exploitation of past long-term traffic patterns, which often form arepetitive and predictable pattern and can be used for a posterioriestimation of future traffic; (b) exploitation of ISP measurement,including those gathered at the backhaul such as broadband traffic,capacity, and neighborhood location information; (c) exploitation ofjoint backhaul capacity and multi-AP schedule design; (d) exploitationof backhaul control points, such as RT in certain DSL backhaul systems,to become a BACK, as an anchor of control plane decisions; and (e)exploitation of location information, geographic maps, and the radioenvironment including radio loss to different locations.

The BACK control plane 399 can also connect to a LTE and/or WiFi gatewayto report the condition of LTE network and enable dynamic choice betweenLTE and WiFi connections. This is particularly likely a scenario ascellular wireless networks continue the trend of reducing cell sizes.The control system, when connected to LTE/WiFi gateway, can also selectthe best backhaul link, with the least congestion and most availablecapacity, for the mix of LTE and WiFi air-interface traffic to be routedto.

With Station-to-Station, peer-to-peer architectures, indirectarchitectures are formed by the STAs resulting in a multihopair-interface STA-STA network. We refer to this as peeringrelationships. The formation of peering relationships is based on thefollowing factors: (a) performance in which some STAs have higher speedconnectivity to APs that also have higher backhaul speed, so called“strong STAs” with the opposite being “weak STAs,” and in which strongSTAs can become peers that help weak STAs; (b) economic in which STAsparticipating in this architectures as helping peers are rewarded eitherthrough monthly bill credits or “tit for tat” strategies; and (c)security in which only those STAs with high level of security, e.g.,strong encryption on messages can use other STAs as relay peers, andonly those STAs with trusted users can act as relay peers.

There are various ways to optimize peering relationships. However, inorder to minimize the overhead, two specific methods are specificallyproposed: reservation and preconfiguration.

Utilizing reservation of a specific peering STA as a one-hop relay whichsignificantly reduces overhead, instability, and packet transitionmechanics associated with dynamically searching for STAs in real time.More generally, considering that some STAs may be powered off inmulti-tenant buildings, each STA has a ranked order list of STAs indescending order of choice as peering STA, with a default length of, forexample “3.” It goes down the list from the first STA on the list, andwhen that is not available, goes to the second, etc.

Utilizing preconfiguration of fixed peering path is done offline basedon performance measurement over a long timescale, e.g., weeks andmonths, and can be updated e.g. every month, or when a peering STA ispowered off continuously for e.g. one week.

For multi-AP access control, control optimization formulation andsolution are proposed. First we introduce our formulation of the problemusing the following notation:

Each STA is indexed by k, with a utility function U_k chosen either byWiFi provider or by consumer;

Each AP is indexed by i, with a backhaul capacity B_i that can be variedby backhaul provider;

S_ki: the fraction of time STA k spends connecting to AP i over a shorttimescale. They must sum up across i to be less than 1 for each k foreach radio interface;

C_i: the capacity region for STAs associated with AP i, which is afunction of all the S_ki;

C_ki: the link capacity from STA k to AP i. The set of C_ki across all kmust lie within the capacity region C_i. The exact tradeoff can becomplicated, depending on many factors in PHY and MAC layers, as well astopologies like the existence of hidden nodes;

X_ki: throughput from STA k to AP i. It is the product of S_ki and C_ki;

The direct optimization variables are S_ki, the scheduling/loadbalancing factors per STA and AP pair. Many of these may be 0. S_ki inturn drive C_ki, which is also influenced by other factors like channelassignment across APs. They collectively determine X_ki;

Then X_ki summed across all k for a given i must be smaller thanbackhaul capacity B_i for AP I; and

X_ki summed across all i for a given k is the input to the utilityfunction for STA k.

TABLE 2 Maximize subject to sum_k U_k (y_k), All k, sum_i X_ki = y_k,All i, sum_k X_ki <= B_i, All (k,i) X_ki = S_ki * C_ki, All i, and sum_iS_ki <= 1, and All i {C_ki}_k in Capacity region C_i({S_ki}_k),

This problem can be solved in various ways, but it can be seen that theextended information and control of a provider-supported control planecan dramatically enhance the efficiency of solving this problem.

This problem can be solved in short timescale approximately (pertimeslot), or over longer timescale for a target equilibrium. Ifbackhaul provider(s) participate in the optimization, then {B_i} becomevariables too. If TM mode is used rather than S mode, S_ki needs to beintegers. Solving this problem amounts to switching APs. In addition tosolving the above problem formulation, we can further impose twoadditional elements to the algorithm:

Solving the above optimization provides a way to choose S_ki, orequivalently, S_k vectors, one for each STA k. This is a short timescaleoptimization.

In the longer timescale optimization, we can also enforce a constraintthat the sum of y_k(t) over a window of timeslots {t} is sufficientlybig, since slower links take longer to complete a job.

Fairness can be controlled through the choice of utility functions U_k.For example, proportional fairness across y_k can be obtained by usinglogarithmic utility functions: U_k=log (y_k). In general, alpha-fairutility functions can be used [12], with larger alpha leading to morefair allocations.

Furthermore, weights can be installed in front of each utility function.For example, U_k=w_k*log(y_k), where weights {w_k} reflect the relativeimportance of STA k. This can be derived from either billing differences(some users pay more), or from multiple QoS classes, includingin-building users, passer-by users, users of various degrees of qualityassurance. These weights can also reflect the number of parallel TCPsessions for a given application flow, as will be further discussed inSection D below.

Another issue important for fairness control is the relationship betweenthe rate that a STA would receive using a single AP and the rate itreceives when using multiple APs. The ratio between these two ratesneeds to be reasonable. There are two ways to incorporate fairness here:(a) instead of looking at the utility function of y_k, we use utilityfunction of this ratio, (b) use a generalized alpha-fair utilityfunction [12] where each STA has a preference parameter q_k, and thisparameter is the normal rate STA k receives without using multiple APs.

This procedure optimizes X_ki, the throughput from STA k to AP i; andS_ki, the scheduling/load balancing factors. The BACK may directly relaythese optimized vectors to each STA and AP, then the equipment assignseach transmission a path and timeslot such that the total trafficmatches X_ki and S_ki as close as possible.

Or, the BACK control plane 399 may indirectly assign link parameters andcapacity to approximate the optimal solution. Differentsource/destination addresses, or different flows (i.e., video streams)may be assigned over different paths. Or a flow may be broken intomultiple fragments, with a tracker file created by the BACK whichdetermines the paths and slots that each data fragment is sent over.

Measurement from STAs is a difficult issue in multi-AP architectures,under practical constraints on the accuracy and granularity ofmeasurements from the STAs. Thus, methods are proposed that use aservice providers capability to run a control plane using a BACK systemto collect data more effectively.

Measurement of backhaul capacity values {B_i} can be carried out throughbackhaul ISP's data and speed tests. This enables connecting to theoptimal APs depending on time-of-day, a long timescale optimization ofS(t).

Measurement of air interface capacity regions {C_i} is made moredifficult because it involves time varying air interface conditions, andin general the capacity regions are coupled when the APs are closeenough together. The BACK collects data, such as the throughput vectorsfor the STAs connected to each AP under different loading conditions, tohelp estimate the capacity regions more accurately. Air interfacecapacity is measured on each link, to each STA. A large database ispopulated, including counts of connection speeds, passive counts ofexisting traffic throughput, and active probing tests measuring delayand throughput.

In both types of measurement above, the invention incorporateshistorical time-of-day data to lessen the need for instantaneousmeasurement. In certain deployment scenarios such as multi-tenantbuildings, data shows that each weekday (other than Friday) exhibitsremarkable repetitive patterns of usage over a 24-hour period, and eachday of the week also exhibits such patterns across different weeks(except for holidays). Using data over a sliding time window, both {B_i}and {C_i} can be approximately predicted ahead of time during each hourof each day.

The optimization and measurement procedure may also be performediteratively to successively decrease error or improve performance.

Joint design of wireless connections and wired backhaul are proposedsuch that air interface and backhaul are be jointly optimized withcompatible devices. The opportunity can be seen by observing in theoptimization problem above that {B_i} constrains the best {X_ki}, thusthe best objective function value, achievable.

However, {B_i} cannot all be increased at the same time. For example, inDSL backhauls, dynamic spectrum management (DSM) methods change thetradeoff among the backhaul links by picking different points on the DSLcapacity region's boundary. Under the joint design in this disclosure,those APs with a higher demand of STA traffic will be given higherpriority in DSM, thus alleviating the bottleneck constraints on thoseAPs. One way to readily tell which AP's capacity to increase is to lookat the optimal Lagrange multipliers or the slackness corresponding toeach of the B_i constraints in the optimization problem. Conversely, ifsome B_i cannot be readily increased further (due to hitting thecapacity region's limit), the STA-AP peering relationship can bere-optimized to avoid passing traffic through that bottleneck.

A related and challenging issue is that of an incentive mechanism toopen WiFi to use by others. Here methods are proposed to leverage“tit-for-tat” mechanisms. For instance, a unit of credit is provided aseach STA or AP opens up to relay traffic over one period of time, e.g.,1 minute. Then over a moving window of e.g. 1 day, each STA and AP needsto have accumulated a minimum amount of credits, e.g. 10 units, in orderto be in a position to participate in multi-AP sharing: asking otherSTAs and APs to help relay its traffic.

A scale can also be built, in which more credits lead to longer periodof time with the “ticket” to participate in multi-AP sharing. In orderto normalize across STAs and APs with different capacities, credits canalso be given proportional to the percentage of relay traffic vs. directtraffic.

The combination of methods described herein effectively lead to multiple“end-to-end” paths between each STA and the boundary of the accessnetwork, e.g., the Broadband Network Gateway (BNG). While the rest ofthe path through the Internet is decided by protocols such as IP andinfluenced by metro and backbone network conditions, the access networkportion described above is often the performance bottleneck. Therefore,control of multi-homing capabilities within the access network is highlyvaluable for instituting optimizations and greater operationalefficiency.

Multi-homing control is valuable in performance tuning, e.g., use of DSLbackhaul traffic data to determine optimal routes. Multi-homing controlis useful for QoS (and revenue base) differentiation, e.g., routedifferent connections, traffic classes, or packets, over differentpaths. Across the access network, the system can also optimize TCPtraffic flows using multiple TCP connections to increase overallbandwidth. This means that we will install fairness and maximizeefficiency across three levels of granularity: per packet, per TCPconnection, and per application flow. This can follow the policy set bya provider or policy manager.

Multi-homing control is also valuable in load balancing, e.g., dynamicassignment of multiple routes. Again, route and time-slot assignment canbe based on historical traffic patterns at different time-of-day andday-of-week. And multi-homing control is valuable in reliability. Themulti-AP architecture effectively enables multi-homing that providesalternative paths in time of severe congestion or equipment failure. Inparticular, node-disjoint paths can be picked out across WiFiair-interface and backhaul so that multiple sessions can share a givennode-disjoint path for backup in time of failure.

FIG. 4 illustrates a diagrammatic representation of a machine 400 in theexemplary form of a computer system, in accordance with one embodiment,within which a set of instructions, for causing the machine/computersystem 400 to perform any one or more of the methodologies discussedherein, may be executed. In alternative embodiments, the machine may beconnected (e.g., networked) to other machines in a Local Area Network(LAN), an intranet, an extranet, or the Internet. The machine mayoperate in the capacity of a server or a client machine in aclient-server network environment, as a peer machine in a peer-to-peer(or distributed) network environment, as a server or series of serverswithin an on-demand service environment. Certain embodiments of themachine may be in the form of a personal computer (PC), a tablet PC, aset-top box (STB), a Personal Digital Assistant (PDA), a cellulartelephone, a web appliance, a server, a network router, switch orbridge, computing system, or any machine capable of executing a set ofinstructions (sequential or otherwise) that specify actions to be takenby that machine. Further, while only a single machine is illustrated,the term “machine” shall also be taken to include any collection ofmachines (e.g., computers) that individually or jointly execute a set(or multiple sets) of instructions to perform any one or more of themethodologies discussed herein.

The exemplary computer system 400 includes a processor 402, a mainmemory 404 (e.g., read-only memory (ROM), flash memory, dynamic randomaccess memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM(RDRAM), etc., static memory such as flash memory, static random accessmemory (SRAM), volatile but high-data rate RAM, etc.), and a secondarymemory 418, which communicate with each other via a bus 430. Main memory404 includes a traffic coordinator 424 and also commands andinstructions. Main memory 404 and its sub-elements (e.g. 423 and 424)are operable in conjunction with processing logic 426 and processor 402to perform the methodologies discussed herein.

Control module 435 is further depicted operable in conjunction withsoftware 422 as well as the traffic coordinator 424 and commands andinstructions 423 as described previously.

Processor 402 represents one or more general-purpose processing devicessuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processor 402 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,processor implementing other instruction sets, or processorsimplementing a combination of instruction sets. Processor 402 may alsobe one or more special-purpose processing devices such as an applicationspecific integrated circuit (ASIC), a field programmable gate array(FPGA), a digital signal processor (DSP), network processor, or thelike. Processor 402 is configured to execute the processing logic 426for performing the operations and functionality which is discussedherein.

The computer system 400 may further include a network interface card408. The computer system 400 also may include a user interface 410 (suchas a video display unit, a liquid crystal display (LCD), or a cathoderay tube (CRT)), an alphanumeric input device 412 (e.g., a keyboard), acursor control device 414 (e.g., a mouse), and a signal generationdevice 416 (e.g., an integrated speaker). The computer system 400 mayfurther include peripheral device 436 (e.g., wireless or wiredcommunication devices, memory devices, storage devices, audio processingdevices, video processing devices, etc.).

The secondary memory 418 may include a non-transitory machine-readableor computer readable storage medium 431 on which is stored one or moresets of instructions (e.g., software 422) embodying any one or more ofthe methodologies or functions described herein. The software 422 mayalso reside, completely or at least partially, within the main memory404 and/or within the processor 402 during execution thereof by thecomputer system 400, the main memory 404 and the processor 402 alsoconstituting machine-readable storage media. The software 422 mayfurther be transmitted or received over a network 420 via the networkinterface card 408.

While the subject matter disclosed herein has been described by way ofexample and in terms of the specific embodiments, it is to be understoodthat the claimed embodiments are not limited to the explicitlyenumerated embodiments disclosed. To the contrary, the disclosure isintended to cover various modifications and similar arrangements as areapparent to those skilled in the art. Therefore, the scope of theappended claims should be accorded the broadest interpretation so as toencompass all such modifications and similar arrangements. It is to beunderstood that the above description is intended to be illustrative,and not restrictive. Many other embodiments will be apparent to those ofskill in the art upon reading and understanding the above description.The scope of the disclosed subject matter is therefore to be determinedin reference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

What is claimed is:
 1. A controller that interfaces with wirelesscommunications nodes, the controller comprising: a first interface tocouple to a first wireless communications node, the first wirelesscommunications node having been established via a first wirelesstransceiver and having access to a first backhaul connection; a secondinterface coupled to a second wireless communications node, the secondwireless communications node having been established via a secondwireless transceiver and having access to a second backhaul connection;and a control module that issues instructions to perform multi-homedload balancing of data traffic through the first and second wirelesscommunications nodes and their respective backhaul connections, theinstructions control at least one of a channel assignment and aconnection assignment among at least one of a network station and anaccess point (AP).
 2. The controller of claim 1, wherein the firstbackhaul connection comprises a Wide Area Network (WAN) backhaulconnection.
 3. The controller of claim 1, wherein a functionality of thecontrol module is performed by a wireless communications device that iscoupled to the first wireless communications node.
 4. The controller ofclaim 1, wherein a functionality of the control module is performed by aserver that is coupled to the first wireless communications node.
 5. Thecontroller of claim 1, wherein the first wireless backhaul connection isa wireless connection.
 6. The controller of claim 1, wherein the firstwireless backhaul connection is one of a wired and a fiber opticbackhaul connection.
 7. The controller of claim 1, wherein theinstructions increase a bandwidth through at least one of the first andsecond backhaul connections.
 8. The controller of claim 1, wherein thecontrol module evaluates a capacity adjustment for an AP to increase ajoint backhaul capacity.
 9. The controller of claim 1, wherein thecontrol module performs a diagnostic on a broadband communicationsservice.
 10. The controller of claim 9, wherein the diagnostic comprisesspeed testing.
 11. The controller of claim 1, wherein a communicationbetween the first interface and the first wireless communications nodesis time-multiplexed.
 12. The controller of claim 1, wherein acommunication between the first interface and the first wirelesscommunications nodes is frequency-multiplexed.
 13. The controller ofclaim 1, wherein the first wireless communications interface is coupledto the first wireless communications node via a multi-hop air-interfacenetwork.
 14. A method interfacing with wireless communications nodes,the method comprising: coupling a first interface to a first wirelesscommunications node, the first wireless communications node having beenestablished via a first wireless transceiver and having access to afirst backhaul connection; coupling a second interface to a secondwireless communications node, the second wireless communications nodehaving been established via a second wireless transceiver and havingaccess to a second backhaul connection; and using a control module toissue instructions to perform multi-homed load balancing of data trafficthrough the first and second wireless communications nodes and theirbackhaul connections, respectively, the instructions controlling atleast one of a channel assignment and a connection assignment among atleast one of a network station and an access point (AP).
 15. The methodof claim 14, comprising evaluating a capacity adjustment for an AP toincrease a joint backhaul capacity.
 16. The method of claim 14,comprising performing a diagnostic on a broadband communicationsservice.
 17. The method of claim 14, comprising determining a parameterassociated with at least one of the first and second wirelesscommunication nodes, the parameter comprising one or more of: a radiolink connection setting; a broadband connection setting; a backhaulconnection at the AP; an Internet Protocol (IP) address assignment for aflow of data packets; an IP address assignment for a first and a secondsub-set of the flow of data packets; a Quality of Service (QoS)classification for the flow of data packets; an assessed signalstrength; and a frequency band.
 18. The method of claim 14, whereincontrolling at least one of a channel assignment and a connectionassignment comprises determining an assessment that is based on one ormore of: a traffic flow; a radio environment; a performance tuningparameter; historic traffic data; historic radio link performance datawithin a geographic map; performance and triangulation data within ageographic Quality of Service (QoS) parameter; information on anunderlying wireless network topology; information on an interference inthe wireless network; a bias toward one or more reliability targets; andrewards or incentives for devices that participate in an aggregated widearea network backhaul connection.
 19. A system to interface withwireless communications nodes, the system comprising: a trafficcoordinator to interface to two or more wireless communications nodes; afirst wireless transceiver that establishes first wirelesscommunications node and has access to a first backhaul connection; asecond wireless transceiver that establishes second wirelesscommunications node and has access to a second backhaul connection;first and second interfaces to couple to the first and second wirelesscommunications nodes, respectively; and a control module comprising aprocessor and a non-transitory computer-readable medium comprisinginstructions that, when executed by the processor, cause steps to beperformed, the steps comprising: issuing instructions to performmulti-homed load balancing of data traffic through the first and secondwireless communications nodes and their respective backhaul connections,the instructions control at least one of a channel assignment and aconnection assignment among at least one of a network station and anaccess point (AP).
 20. The system of claim 19, wherein the controlmodule determines a parameter associated with at least one of the firstand second wireless communication nodes, the parameter comprising one ormore of: a radio link connection setting; a broadband connectionsetting; a backhaul connection at the AP; an Internet Protocol (IP)address assignment for a flow of data packets; an IP address assignmentfor a first and a second sub-set of the flow of data packets; a Qualityof Service (QoS) classification for the flow of data packets; anassessed signal strength; and a frequency band.